Image processing device that generates and selects between multiple image signals based on zoom selection

ABSTRACT

An image processing device including a first lens, a second lens disposed on one side of the first lens, a third lens disposed on the other side of the first lens, a first image sensor which receives an input of a first image obtained from the first lens to generate a first image signal, a second image sensor which receives an input of a second image obtained from the second lens to generate a second image signal, a third image sensor which receives an input of a third image obtained from the third lens to generate a third image signal, a selector which receives the input of the second image signal and the third image signal, outputs the second image signal under a first condition, and outputs the third image signal, under a second condition different from the first condition and an image processor which performs image processing, using the first image signal and an output signal of the selector.

This is a Continuation of U.S. application Ser. No. 15/844,205, filedDec. 15, 2017, now U.S. Pat. No. 10,348,978 issued on Jul. 9, 2019,which claims priority from Korean Patent Application No. 10-2017-0069040filed on Jun. 2, 2017, the subject matter of which is herebyincorporated herein by reference.

BACKGROUND

The present inventive concept relates to processors, image processingdevices including a processor, and methods of processing an image usingan image processing device.

Certain image processing devices may include multiple camera modulesrespective images, collectively referred to as multiple images.Accordingly, such image processing devices may process multiple imagesto generate a single image.

In order to acquire a relatively high quality image, a given cameramodule may include a dual sensor. A camera module including a dualsensor may include, for example, a wide angle lens and a telephoto lens,where the combined capabilities of the wide angle lens and telephotolens may advantageously be used during zoom operations, for example,performed in a relatively high illuminance environment. However, this isnot the case in relatively low illuminance environments, where lightlevels are insufficient to realize the benefits of the telephoto lens.

SUMMARY

An aspect of the present inventive concept provides a processor capableof improving the performance of a zooming operation when the illuminanceis high, an image processing device including the processor, and amethod for processing image.

Another aspect of the present inventive concept provides a processorcapable of improving image quality even when the illuminance is low, animage processing device including the processor, and a method forprocessing image.

Still another aspect of the present inventive concept provides aprocessor capable of reducing a rectification error on the imagesobtained by each lens, an image processing device including theprocessor, and a method for processing image.

The aspects of the present inventive concept are not limited to thosementioned above and another technical problem which has not beenmentioned can be clearly understood by those skilled in the art from thedescription below.

According to some embodiments of the present inventive concept, there isprovided an image processing device comprising a first lens, a secondlens disposed on one side of the first lens, a third lens disposed onthe other side of the first lens, a first image sensor which receives aninput of a first image obtained from the first lens to generate a firstimage signal, a second image sensor which receives an input of a secondimage obtained from the second lens to generate a second image signal, athird image sensor which receives an input of a third image obtainedfrom the third lens to generate a third image signal, a selector whichreceives the input of the second image signal and the third imagesignal, outputs the second image signal under a first condition, andoutputs the third image signal, under a second condition different fromthe first condition and an image processor which performs imageprocessing, using the first image signal and an output signal of theselector.

According to some embodiments of the present inventive concept, there isprovided a processor comprising an image processor which receivesprovision of a first image signal from a first camera module and a firstselector which receives provision of a second image signal from a secondcamera module, receives provision of a third image signal from a thirdcamera module, outputs the second image signal under a first condition,and outputs the third image signal under a second condition differentfrom the first condition, wherein the image processor performs imageprocessing, using the first image signal and an output signal of thefirst selector.

According to some embodiments of the present inventive concept, there isprovided a method for image processing comprising receiving a firstimage signal from a first camera module by a first signal processor toperform first image signal processing, receiving a second image signalfrom a second camera module and receiving a third image signal from athird camera module by a selector to output the second image signalunder a first condition and output the third image signal under a secondcondition different from the first condition and generating an outputimage by an image processor, using the output signal of the first signalprocessor and the output signal of the selector.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects and features of the present inventiveconcept will become more apparent upon consideration of certainexemplary embodiments thereof with reference to the attached drawings,in which:

FIG. 1 is a block diagram illustrating an image processing device 10according to some embodiments of the inventive concept;

FIGS. 2A and 2B are respective views illustrating exemplaryconfigurations for the first, second and third lenses of FIG. 1;

FIG. 3 is a block diagram further illustrating in one embodiment thefirst signal processor 310 of FIG. 1;

FIG. 4 is a block diagram further illustrating in one embodiment thesecond signal processor 320 of FIG. 1;

FIG. 5 is a block diagram further illustrating in one embodiment thefirst image generator 400 of FIG. 1;

FIG. 6 is a block diagram further illustrating in one embodiment thedepth information generator 410 of FIG. 5;

FIG. 7 is a block diagram further illustrating in one embodiment thefirst output image generator 420 of FIG. 5;

FIG. 8 is a block diagram further illustrating in one embodiment thesecond output image generation unit 430 of FIG. 5;

FIG. 9 is another block diagram illustrating an image processing deviceaccording to some embodiments of the inventive concept;

FIG. 10 is a block diagram further illustrating in one embodiment thethird signal processor 330 of FIG. 9;

FIGS. 11 and 12 are respective block diagrams of an image processingdevice according to still other embodiments of the inventive concept;

FIG. 13 is a block diagram further illustrating the second imagegenerator 500 of FIG. 12;

FIG. 14 is a flowchart summarizing method steps that may be performed byvarious selector elements according to some embodiments of the inventiveconcept;

FIG. 15 is a general block diagram of an image system that may beconfigured to include an processing device according to some embodimentsof the inventive concept; and

FIG. 16 is a general block diagram illustrating part of an image systemincluding an image processing device according to some embodiments ofthe inventive concept.

DETAILED DESCRIPTION

Certain image processing devices according to embodiments of theinventive concept will be now be described in some additional detailwith reference to FIGS. 1 to 8.

FIG. 1 is a block diagram illustrating an image processing deviceaccording to embodiments of the inventive concept, where the imageprocessing device generally includes a camera module 1110 and aprocessor 1120.

Image signals (e.g., is1, is2, and is3) respectively generated by thecamera module 1110 are provided to the processor 1120, such that theprocessor 1120 may process the one or more of the image signals (e.g.,is1, is2, and is3) in a manner that generates the output image io. Inthe illustrated embodiments, various conventionally understood elementsmay be optionally incorporated between the camera module 1110 andprocessor 1120. For example, the respective image signals (e.g., is1,is2, and is3) generated by the camera module 1110 may be provided to theprocessor 1120 via certain constituent elements and/or interfacesdisposed between the camera module 1110 and processor 1120.

In the illustrated embodiment of FIG. 1, the camera module 1110 includesa first camera module 1111, a second camera module 1112, and a thirdcamera module 1113, where the first camera module 1111 includes a firstlens L and a first image signal output unit 1010; the second cameramodule 1112 includes a second lens L2 and a second image signal outputunit 1020; and the third camera module 1113 includes a third lens L3 anda third image signal output unit 1030.

Here, at least one of the first lens L and third lens L3 may be adifferent type of lenses when compared to the second lens L2. Forexample, the first lens L1 and third lens L3 may be wide angle lenses,and the second lens L2 may be a telephoto lens.

FIGS. 2A and 2B are respective views illustrating exemplaryconfigurations of the first L1, second L2, and third L3 lenses of FIG.1.

Referring to FIG. 2A, the first lens L1, second lens L2 and third lensL3 are horizontally disposed across one side surface (e.g., a front sideor a rear side) of the body of an electronic device 1000 (e.g., a mobilephone). In certain configurations, the second lens L2 is disposed on oneside of the first lens L1 while the third lens L3 is disposed on theopposing (or other) side of the first lens L1. In FIG. 2A, the secondlens L2 and third lens L3 are disposed symmetrically about the firstlens L1. That is, assuming that the second lens L2 is spaced apart fromthe first lens L1 on one side by a first distance d1 and the third lensL3 is spaced apart from the first lens L1 on the other side by a seconddistance d2, the illustrated embodiment of FIG. 2A assumes that thefirst distance d1 and second distance d2 are substantially the same.However, this need not always be the case in other embodiments of theinventive concept.

In the illustrated embodiment of FIG. 2A, centers of the first lens L1,second lens L2, and third lens L3 are linearly disposed along horizontalline, assuming for purposes of this description an arbitrary definitionof the term “horizontal” as meaning a direction across the width of theelectronic device 1000. Using this assumed definition, the illustratedembodiment of FIG. 2B vertically disposes the centers of the first lensL1, second lens L2 and third lens L3 along a substantially same verticalline.

Referring to FIGS. 2A and 2B, when an object is disposed to face thefirst lens L1, the subject and the first lens L1 are arranged inalignment with each other, and the first distance d1 and the seconddistance d2 are different from each other, the angles at which light isreflected from the subject is incident to each of the first lens L1, thesecond lens L2 and the third lens L3 will be different from each other.

Therefore, when accurately representing the subject, uncompensated (orunrectified) distortion may be present in the images (e.g., id1, id2,and id3) obtained from each of the first lens L1, the second lens L2 andthe third lens L3. That is, since the first distance d1 and seconddistance d2 are different from each other, some degree of distortionbetween the images (e.g., id1, id2, and id3) may become severe.

In order to correct the foregoing image distortion, a rectificationoperation may be performed on each of the images (e.g., id1, id2, andid3) obtained from each of the first lens L1, second L2 and third lensL3. At this time, since the degree of distortion between the images(e.g., id1, id2, and id3) may be severe, errors of the rectificationoperation may be correspondingly large. And if the resultingrectification operation error is large, the noise (artefact) of theoutput image (e.g., io) may become large.

In image processing devices, like the image processing device of FIG. 1,the second lens L2 and the third lens L3 may be respectively spacedapart from the first lens L at substantially the same interval (e.g.,first distance d1=second distance d2). And where the first distance d1is substantially the same as the second distance d2, the degree ofdistortion between the images (e.g., id1, id2, and id3) may besignificantly reduced over configurations wherein the first distance d1and second distance d2 are substantially different. Therefore, withrespect to rectification operation(s) subsequently performed on therespective images (e.g., id1, id2, and id3), the rectification error maybe reduced.

Referring back to FIG. 1, the first image signal output unit 1010 mayinclude, for example, a first image sensor 1011, a first storage device1012, and a first analog-to-digital converter 1013; the second imagesignal output unit 1020 may include, for example, a second image sensor1021, a second storage device 1022, and a second analog-to-digitalconverter 1023; and the third image signal output unit 1030 may include,for example, a third image sensor 1031, a third storage device 1032, anda third analog-to-digital converter 1033.

Each of the first image id1, the second image id2, and the third imageid3 is respectively obtained from the first lens L1, second lens L2 andthird lens L3. The first to third images id1, id2, and id3 may then berespectively provided to the first output unit 1010, second output unit1020, and third output unit 1030. And each of the first, second andthird image sensors 1011, 1021, and 1031 may be used to respectivelysense the first, second and third images id1, id2, and id3.

Hence, the first image sensor 1011 (e.g., a color image sensor) receivesthe first image id1 to generate the first image signal is1; the secondimage sensor 1021 (e.g., a color sensor) receives the second image id2to generate the second image signal is2; and the third image sensor 1031(e.g., a monochrome image sensor) receives the third image id3 togenerate the third image signal is3.

Calibration data of each of the first, second and third camera modules1111, 1112, and 1113 may be respectively stored in the first, second andthird storage devices 1012, 1022, and 1032, where the term “calibrationdata” may include, for example, information associated with a degree ofrotation for a corresponding camera module, focal length, and/or opticalaxis.

Each of the first, second and third analog-to-digital converters 1013,1023, and 1033 may be used to respectively convert the format of thefirst, second and third image signals id1, id2, and id3. For example, itis possible to respectively convert the first, second and third imagesignals id1, id2, and id3 having an analog format into a correspondingdigital signal (i.e., first image signal is, second image signal is2,and third image signals is3).

The first, second and third image signals is1, is2, and is3 respectivelyprovided by the first, second and third camera modules 1111 to 1113 maybe input to the processor 1120. In the illustrated embodiment of FIG. 1,the processor 1120 includes first, second and third interfaces 110, 120,and 130, a first selector 210, a first signal processor 310, a secondsignal processor 320, and a first image generator 400. However, this isjust one of many particular arrangement of elements and/or functionsthat may be used to implement a processor in various embodiments of theinventive concept.

However, in the illustrated embodiment of FIG. 1, the first, second andthird interfaces 110, 120, and 130 may respectively be used to changeone or more signal formats compatible with the first, second and thirdimage sensors 1011, 1021 and 1031 to a format more compatible theprocessor 1120. Here, the first, second and third interfaces 110, 120,and 130 may be, for example, a camera serial interface (CSI).

In FIG. 1, the first image signal is1 is provided to the first signalprocessor 310 via the first interface 110; the second image signal is2is provided to the first selector 210 via the second interface 120; andthe third image signal is3 is provided to the first selector 210 via thethird interface 130. The first selector 210 receives, as inputs, thesecond image signal is2 and the third image signal is3, and in responseto a selection signal SS, the first selector 210 outputs a selected oneof the second image signal is2 and the third image signal is3, as afirst image output signal isa provided to the image processor 1121. Insome embodiments, the first selector 210 may be disposed separately fromthe second signal processor 320 of the image processor 1121.

Using the foregoing embodiment as a working example, the selectionsignal SS may be configured in relation to one of a first condition C1(e.g., a logic value of ‘0’)—selecting the second image signal is2, or asecond condition C2 (e.g., a logic value of ‘1’)—selecting the thirdimage signal is3. In this regard, the first condition C1 and secondcondition C2 may be different conditions. For example, the firstcondition C1 may indicate a high illuminance environment (i.e.,illuminance greater than a predetermined value) and the second conditionC2 may indicate a low illuminance environment (i.e., illuminance lessthan the predetermined value). In this context, the term “illuminance”is used to denote a brightness of a location associated with an imagedsubject as obtained (e.g.,) from the first lens L1 of the first cameramodule 1111.

In some embodiments, the first signal processor 310 may be used to senseilluminance and communicate corresponding illuminance information to theselection signal generator 310_SS, where the selection signal generator310_SS is used to generate the selection signal SS in response to theilluminance information. In the illustrated embodiment of FIG. 1, theselection signal generator 310_SS is shown as being disposed in theprocessor 1120, but this need not always be the case in otherembodiments of the inventive concept. For example, the selection signalgenerator 310_SS may be a hardware component disposed external to theprocessor 1120, or a software component providing the selection signalgeneration function.

In FIG. 1, the image processor 1121 of the processor 1120 receives thefirst image signal is1, as well as the combination of the output signalisa of the first selector 210 and the selection signal SS in order togenerate the output image io. Here, the image processor 1121 includesthe first signal processor 310, second signal processor 320, and firstimage generator 400.

The first signal processor 310 receives the first image signal is 1 asan input and performs first image signal processing to provide a firstimage signal is1 a to the first image generator 400. In thisconfiguration, for example, the first image signal is1 a provided by thefirst signal processor 310 may be the first output signal is1 subjectedto a first image signal processing by the first signal processor 310.

FIG. 3 is a block diagram further illustrating in one embodiment thefirst signal processor 310 of FIG. 1.

Referring to FIG. 3, the first signal processor 310 may include a firstformat converter 311, a first noise removal unit 313, a first imageenhancer 315, and a second format converter 317. The first imageprocessing performed by the first signal processor 310 may includesequentially passing the first image signal is1 through the first formatconverter 311, first noise removal unit 313, first image enhancer 315,and second format converter 317 in order to generate the first outputsignal is1 a having an appropriate signal format.

That is, the first format converter 311 may receive the first imagesignal is 1, change its format and provide the format-changed firstimage signal to the first noise removal unit 313. In this regard, thefirst image signal is 1 may have a format defined in accordance with aspecific form of color filter arrangement used by the first cameramodule 1111. For example, the first format converter 311 may convert theformat of the first image signal is 1 to the conventionally understoodRGB format in circumstances where, for example, the first image sensor1011 of the first camera module 1111 includes a Bayer color arrangement.In this case, the first format converter 311 may convert the format ofthe first image signal is 1 to the RGB format. The first formatconverter 311 may, for example, perform a demosaic operation. In otherembodiments, operation of the first format converter 311 may be omittedaccording to the type of color filter of the first image sensor 1011 ofthe first camera module 1111.

The first noise removal unit 313 may be used to remove one or more noisecomponents from the output signal provided by the first format converter311, and provide the resulting noise-filtered output signal to the firstimage enhancer 315.

In turn, the first image enhancer 315 receives the noise-filtered outputsignal from the first noise removal unit 313 performs signal processingthat improves the quality of the resulting signal, and provides thequality-improved output signal to the second format converter 317. Forexample, the first image enhancer 315 may be used to remove a falsecolor component from the noise-filtered output signal provided by thefirst noise removal unit 313, or may be used to perform a sharpeningoperation, or the like.

The second format converter 317 receives the quality-improved signalprovided by the first image enhancer 315 and may be used to convert theformat of this signal into a format compatible with the first imagegenerator 400. For example, the quality-improved signal provided by thefirst image enhancer 315 may have a RGB format that is converted to(e.g.,) a YUV format including luminance information and colorinformation using the second format converter 317.

Referring again back to FIG. 1, the second signal processor 320 receivesthe first image output signal isa from the first selector 210 andperforms a second image signal processing in order to generate a secondimage output signal isb that is format compatible with the first imagegenerator 400.

FIG. 4 is a block diagram further illustrating in one embodiment thesecond signal processor 320 of FIG. 1.

Referring to FIG. 4, the second signal processor 320 may include asecond selector 220, a third format converter 321, a third selector 230,a second noise removal unit 323, a second image enhancer 325, a fourthselector 240, a fourth format converter 327, and a fifth selector 250.The second image signal processing performed by the second signalprocessor 320 may include generation of a second image output signal isbhaving a desired format with respect to the first image output signalisa received from the first selector 210. The selection signal SSprovided to the first selector 210 may also be input to the second tofifth selectors 220, 230, 240, and 250.

When the selection signal SS includes information corresponding to thefirst condition C1, the first image output signal isa of the firstselector 210 may be the second image signal is2. In this case, thesecond selector 220 may provide the second image signal is2 to the thirdformat converter 321 on the basis of the selection signal SS.

The third format converter 321 may receive the input of the second imagesignal is2 and convert the format of the second image signal is2 toprovide the second image signal to the third selector 230. The thirdformat converter 321 may be substantially the same as the first formatconverter 311 (see FIG. 3) of the first signal processor 310.

Under the first condition C1, the third selector 230 may provide theoutput signal of the third format converter 321 to the second noiseremoval unit 323, based on the selection signal SS.

The second noise removal unit 323 may receive the input of the outputsignal of the third selector 230 and remove the noise of the outputsignal of the third selector 230 to provide the output signal to thesecond image enhancer 325. The second noise removal unit 323 may besubstantially the same as the first noise removal unit 313 (see FIG. 3)of the first signal processor 310.

The second image enhancer 325 may receive the input of the output signalof the second noise removal unit 323 and execute the work for improvingthe quality of the second image signal is2 to output the output signalto the fourth selector 240. The second image enhancer 325 may besubstantially the same as the first image enhancer 315 (see FIG. 3) ofthe first signal processor 310.

Under the first condition C1, the fourth selector 240 may provide theoutput signal of the second image enhancer 325 to the fourth formatconverter 327, based on the selection signal SS.

The fourth format converter 327 may convert the format of the outputsignal of the second image enhancer 325 to provide the output signal tothe first image generator 400. The fourth format converter 327 may besubstantially the same as the second format converter 317 (FIG. 3) ofthe first signal processor 310.

When the selection signal SS includes information corresponding to thesecond condition C2, the output signal isa of the first selector 210 maybe the third image signal is3. In this case, the second selector 220 andthe third selector 230 may provide the third image signal is3 to thesecond noise removal unit 323, based on the selection signal SS. Thethird image sensor 1031 of the third camera module 1113 may be a blackand white image sensor. Therefore, the format conversion work using thethird format converter 321 may be unnecessary for the third image signalis3.

The second noise removal unit 323 may receive the input of the thirdimage signal is3 and remove the noise of the third image signal is3 toprovide the third image signal to the second image enhancer 325. Thesecond image enhancer 325 may receive the output signal of the secondnoise removal unit 323 and execute the work for improving the quality ofthe output signal of the second noise removal unit 323 to provide theoutput signal to the fourth selector 240.

Under the second condition C2, the fourth selector 240 may provide theoutput signal of the second image enhancer 325 to the fifth selector250, based on the selection signal SS. Under the second condition C2,the fifth selector 250 may output the output signal of the second imageenhancer 325 as the output signal isb, based on the selection signal SS.That is, the format conversion work using the fourth format converter327 may be unnecessary for the third image signal is3.

Referring again to FIG. 1, the first image generator 400 may perform theimage processing and generate the output image io, using the outputsignal is1 a of the first signal processor 310 and the output signal isbof the second signal processor 320.

FIG. 5 is a block diagram further illustrating in one embodiment thefirst image generator 400 of FIG. 1.

Referring to FIG. 5, the first image generator 400 includes a depthinformation generation unit 410, a first output image generation unit420, a second output image generation unit 430, and a third output imagegeneration unit 440. In some embodiments, the image processing performedby the first image generator 400 may include generation of the two imagesignals isb and is1 a as a single image (e.g., io1 or io2), using thedepth information generation unit 410, the first output image generationunit 420, the second output image generation unit 430, and the thirdoutput image generation unit 440.

The depth information generation unit 410 may receive the input of thetwo image signals isb and is1 a and output the depth information is_410a and the corresponding disparity information is_410 b.

FIG. 6 is a block diagram further illustrating in one embodiment thedepth information generation unit 410 of the first image generator 400of FIG. 5.

Referring to FIG. 6, the depth information generation unit 410 mayinclude an adjusting unit 411, a corresponding disparity informationgenerator 413, and a depth information generator 415.

Each of the two image signals isb and is1 a may be input to theadjusting unit 411. Two adjusting units 411 may be disposed tocorrespond to the two image signals isb and is1 a. The adjusting unit411 may perform a rectification work for correcting the distortion ofeach of the two image signals isb and is1 a. That is, the adjusting unit411 may rectify the two image signals isb and is1 a so as to berow-aligned. In the course of the rectification work, the adjusting unit411 may use, for example, calibration data and the like stored in thefirst to third storage devices 1012, 1022, and 1032 of the first tothird camera modules 1111, 1112, and 1113.

The respective outputs of the adjusting unit 411 may be input to thecorresponding disparity information generator 413. The correspondingdisparity information generator 413 may search corresponding points ineach of the output signals of the adjusting unit 411, and may calculatethe corresponding disparity of the output signal of the adjusting unit411 in which the two image signals isb and is1 a are rectified. Thecorresponding disparity information generator 413 may output thecorresponding disparity information is_410 b. The correspondingdisparity information is_410 b may be, for example, a correspondingdisparity map.

The corresponding disparity information is_410 b may be provided to thedepth information generator 415. The depth information generator 415 mayconvert the corresponding disparity information is_410 b by thedisparity to generate the depth information is_410 a. The depthinformation is_410 a may be, for example, a depth map.

Referring again to FIG. 5, the first output image generation unit 420may receive the input of the output signal is1 a and depth informationis_410 a of the first signal processor 310 to generate the first outputimage io1.

FIG. 7 is a block diagram illustrating in one embodiment the firstoutput image generation unit 420 of the first image generator 400 ofFIG. 5.

Referring to FIG. 7, the first output image generation unit 420 mayinclude an out-focus unit 421. The out-focus unit 421 may convert theoutput signal is1 a of the first signal processor 310 into the firstoutput image io1, using the depth information is_410 a. The out-focusunit 421, for example, may provide an out-focusing effect to the outputsignal is1 a of the first signal processor 310.

Referring again to FIG. 5, the second output image generation unit 430may receive the input of the two image signals isb and is1 a and thecorresponding disparity information is_410 b to generate a second outputimage io2

FIG. 8 is a block diagram further illustrating in one embodiment thesecond output image generation unit 430 of the first image generator 400of FIG. 5.

Referring to FIG. 8, the second output image generation unit 430 mayinclude a pixel mapping unit 431 and a mixing unit 433. The pixelmapping unit 431 may generate a pixel map of the two image signals isband is1 a, using the corresponding disparity information is_410 b. Themixing unit 433 may receive the input of the two image signals isb andis1 a and mix the two image signals isb and is1 a using the output ofthe pixel mapping unit 431 to generate a single second output image io2.

Referring again to FIG. 5, when the selection signal SS includesinformation corresponding to the first condition C1, the third outputimage generation unit 440 may scale one of the output signal is1 a ofthe first signal processor 310 and the output signal isb of the secondsignal processor 320 to generate the third output image io3. Forexample, when the processor (1120 of FIG. 1) is an application processorAP built in the cellular phone and the selection signal (SS of FIG. 1)includes information corresponding to the first condition (C1 of FIG.1), a zoom mode may be supported for the user.

When the scale ratio is low (i.e., in a third condition C3), the outputsignal is1 a of the first signal processor 310 may be provided to theoutput image generation unit 440 by the zoom selector 441. The thirdoutput image generation unit 440 may scale the output signal is1 a ofthe first signal processor 310 to generate a third output image io3.That is, when the zoom selection signal SS_ZOOM includes informationcorresponding to the third condition C3, the zoom selector 441 mayprovide the output signal is1 a of the first signal processor 310 to thethird output image generation unit 440. Here, the third condition C3 maybe, for example, a case wherein the angle of view associated with thethird output image io3 generated by the third output image generationunit 440 is less than the angle of view associated with the second image(id2 of FIG. 1).

When the scale ratio increases and the angle of view associated with thethird output image io3 generated by the third output image generationunit 440 is greater than or equal to the angle of view associated withthe second image (id2 of FIG. 1) (i.e., in a fourth condition C4), thezoom selector 441 may output the output signal isb of the second signalprocessor 320 to the third output image generation unit 440. The thirdoutput image generation unit 440 may scale the output signal isb of thesecond signal processor 320 to generate a third output image io3. Thatis, when the zoom selection signal SS_ZOOM includes informationcorresponding to the fourth condition C4, the zoom selector 441 mayoutput the output signal isb of the second signal processor 320 to thethird output image generation unit 440. At this time, the fourthcondition C4 may be, for example, a case where the angle of view of thethird output image io3 generated by the third output image generationunit 440 is equal to or larger than the angle of view of the secondimage (id 2 of FIG. 1).

The zoom selection signal SS_ZOOM may be determined by the user whoselects the scale ratio, for example, when the processor (1120 ofFIG. 1) is an application processor AP built in the mobile phone. Forexample, the zoom selection signal SS_ZOOM may be generated by the zoomselection signal generator. The zoom selection signal generator maygenerate the zoom selection signal SS_ZOOM, using information on thescale ratio selected by the user. The zoom selection signal generatormay be provided as software in some embodiments.

Referring again to FIG. 1, the output image io which is the output ofthe first image generator 400 may be one of the first output image io1and the second output image io2. For example, one of the first outputimage io1, the second output image io2, and the third output image io3may be output as the output image io in accordance with the mode of thepredetermined image system. For example, when the processor 1120 is anapplication processor AP built in the cellular phone, one of the firstoutput image io1, the second output image io2 and the third output imageio3 may be output to the liquid crystal screen of the mobile phone.

Hereinafter, an image processing device according to some embodiments ofthe present inventive concept will be described with reference to FIG. 9and FIG. 10.

FIG. 9 is a block diagram illustrating an image processing deviceaccording to some embodiments of the inventive concept, and FIG. 10 is ablock diagram further illustrating in one embodiment the third signalprocessor 330 of FIG. 9.

Referring to FIGS. 9 and 10, the third signal processor 330 may includea sixth selector 260 which outputs one of the second image signal is2 orthe third image signal is3. The third signal processor 330 may furtherinclude a third format converter 321, a second noise removal unit 323, asecond image enhancer 325, a seventh selector 270, a fourth formatconverter 327, and an eighth selector 280. The selection signal SS whichis input to the sixth through eighth selectors 260, 270, and 280 may beinput to the third signal processor 330.

The third signal processor 330 may receive the input of the second imagesignal is2 and the third image signal is3. The second image signal is2may be input to the sixth selector 260 via the third format converter321. The third image signal is3 may be input to the sixth selector 260.

The sixth selector 260 may output the output signal of the third formatconverter 321 based on the selection signal SS under the first conditionC1. In addition, the sixth selector 260 may output the third imagesignal is3 based on the selection signal SS under the second conditionC2. In other words, the sixth selector 260 may perform the same functionas the first selector (210 of FIG. 1) described with reference to FIG.1.

The output signal isa of the sixth selector 260 may be input to theseventh selector 270 via the second noise removal unit 323 and thesecond image enhancer 325.

The seventh selector 270 may connect the input terminal of the fourthformat converter 327 and the input terminal of the second image enhancer325, based on the selection signal SS under the first condition C1.Meanwhile, the seventh selector 270 may output the output signal of thesecond image enhancer 325 as the output signal isb of the third signalprocessor 330, based on the selection signal SS under the secondcondition C2.

FIG. 11 is a block diagram of an image processing device according tosome embodiments of the inventive concept.

Referring to FIG. 11, the camera module 1110 1 may include a ninthselector 290. The ninth selector 290 may select either the second imagesignal is2 or the third image signal is3 and output the image signal asthe output signal isa. The ninth selector 290 may perform substantiallythe same function as the first selector (210 of FIG. 1) described withreference to FIG. 1.

The camera module 1110 1 may provide the first image signal is1 and theoutput signal isa of the ninth selector 290 to the processor 1120. Theprocessor 1120 may include a first interface 110, a second interface120, and an image processor 1121. The image processor 1121 may include afirst signal processor 310, a second signal processor 320, and a firstimage generator 400.

FIG. 12 is a block diagram illustrating an image processing deviceaccording to some embodiments of the inventive concept, and FIG. 13 is ablock diagram further illustrating in one embodiment the second imagegenerator 500 of FIG. 12.

Referring to FIGS. 12 and 13, the image processor 1121 may receive theinput of the first through third image signals is1 and is3. The imageprocessor 1121 may include a first signal processor 310, a fourth signalprocessor 340, a fifth signal processor 350, and a second imagegenerator 500. The second image generator 500 may include a tenthselector 299 that receives the output signal is2 a of the fourth signalprocessor 340 and the output signal is3 a of the fifth signal processor350 to output one of the output signal is2 a of the fourth signalprocessor 340 and the output signal is3 a of the fifth signal processor350 on the basis of the selection signal SS.

The fourth signal processor 340 may receive the second image signal is2passed through the second interface 120 and execute the first imagesignal processing to output the output signal isa. The fifth signalprocessor 350 may receive the third image signal is3 passed through thethird interface 130 and execute the first image signal processing tooutput the output signal is3 a. The fourth signal processor 340 and thefifth signal processor 350 may be substantially the same as the firstsignal processor (310 of FIG. 1) described with reference to FIGS. 1 and3.

The output signal is1 a of the first signal processor 310, the outputsignal is2 a of the fourth signal processor 340, and the output signalis3 a of the fifth signal processor 350 may be provided to the secondimage generator 500.

The tenth selector 299 included in the second image generator 500 mayperform substantially the same function as the first selector 210described with reference to FIG. 1. Therefore, the output signal isaunder the first condition C1 may be the output signal is2 a of thefourth signal processor 340, and the output signal isa under the secondcondition C2 may be the output signal is3 a of the fifth signalprocessor 350.

The depth information generation unit 410 may receive the input of theoutput signal isa of the tenth selector 299 and the output signal is1 aof the first signal processor 310 to generate the depth informationis_410 a and the corresponding disparity information is_410 b. The firstoutput image generation unit 420 may receive the input of the outputsignal is1 a of the first signal processor 310 and the depth informationis_410 a to generate the first output image io1. The second output imagegeneration unit 430 may receive the input of the output signal is1 a ofthe first signal processor 310, the output signal isa of the tenthselector 299, and the corresponding disparity information is_410 b togenerate the second output image io2. The third output image generationunit 440 may generate the third output image io3, using the outputsignal of the zoom selector 441.

Hereinafter, an image processing method according to some embodiments ofthe present inventive concept will be described with reference to FIG.14. For the sake of clarity of explanation, repeated descriptions ofthose described above will not be provided.

FIG. 14 is a flow chart summarizing certain method steps that may beperformed by any one of the first selector (210 of FIG. 1), the sixthselector (260 of FIG. 10), the ninth selector (290 of FIG. 11), and thetenth selector (299 of FIG. 13) according to some embodiments of theinventive concept.

Referring to FIG. 14, a second image signal and a third image signal maybe provided (S100).

Specifically, for example, the selector (210 in FIG. 1, 260 in FIG. 10,290 in FIG. 11, and 299 in FIG. 13) may receive the provision of thesecond image signal is2 generated from the second camera module (1112 inFIGS. 1, 9, 11, and 12), and the third image signal is3 generated fromthe third camera module (1113 in FIGS. 1, 9, 11, and 12).

A comparison between the illuminance and the predetermined value is made(S200). When the illuminance is greater than or equal to thepredetermined value (i.e., the first condition C1), the selector (210 inFIG. 1, 260 in FIG. 10, 290 in FIG. 11, and 299 in FIG. 13) may outputthe second image signal is2. When the illuminance is less than thepredetermined value (i.e., the second condition C2), in step S400, theselector (210 in FIG. 1, 260 in FIG. 10, 290 in FIG. 11, and 299 in FIG.13) may output the third image signal is3.

The image processor 1121 may include generation of the output image io,using the first image signal is1 and the output signal of the selector(210 in FIG. 1, 260 in FIG. 10, 290 in FIG. 11, and 299 in FIG. 13).

An image processing device according to embodiments of the inventiveconcept may include a selector (210 in FIG. 1, 260 in FIG. 10, 290 inFIG. 11, and 299 in FIG. 13) which selects and outputs one of the secondimage signal is2 and the third image signal is3.

In the case of the first condition C1, the selector (210 in FIG. 1, 260in FIG. 10, 290 in FIG. 11, and 299 in FIG. 13) may select and outputthe second image signal is2 (or the output signal is2 a of the fourthsignal processor (340 in FIG. 12)). The image processor 1121 maygenerate the output image io, using the first image signal is1 generatedfrom the first camera module (1111 in FIGS. 1, 9, 11 and 12) and thesecond image signal is2 generated from the second camera module (1112 inFIGS. 1, 9, 11 and 12). The first lens L1 of the first camera module(1111 in FIGS. 1, 9, 11 and 12) is a wide angle lens, and the secondlens L2 of the second camera module (1112 in FIGS. 1, 9, 11 and 12) maybe a telephoto lens. Therefore, when the illuminance is relatively high,the zoom operation performance of the image processing device may beimproved.

In the case of the second condition C2, the selector (210 of FIG. 1, 260of FIG. 10, 290 of FIG. 11, and 299 of FIG. 13) may select and outputthe output signal of the third image signal is3 (or the output signalis3 a of the fifth processor (350 in FIG. 12)). The processor 1120 maygenerate the output image io, using the first image signal is 1generated from the first camera module (1111 in FIGS. 1, 9, 11 and 12)and the third image signal is3 generated from the third camera module(113 in FIGS. 1, 9, 11, and 12). The first image sensor 1011 of thefirst camera module (1111 in FIGS. 1, 9, 11 and 12) may be a color imagesensor, and the third image sensor of the third camera module (1113 inFIGS. 1, 9, 11 and 12) may be a black and white image sensor. When theilluminance is relatively low, the black and white image sensor may havehigher sensitivity of the image than the color image sensor. Therefore,when the illuminance is relatively low, the quality of the output imageio of the image processing device may be improved.

FIG. 15 is a general block diagram of an image system including theimage processing device according to some embodiments of the inventiveconcept. FIG. 16 is a general block diagram illustrating part of animage system including the image processing device according to someembodiments of the inventive concept.

Referring to FIG. 15, the image system 2000 may include a camera module1110, a processor 1120, a storage device 1130, an interface 1140 and abus 1150. The camera module 1110, the processor 1120, the storage device1130 and/or the interface 1140 may be interconnected via the bus 1150.The bus 1150 may correspond to a path through which data is transferredbetween the camera module 1110, the processor 1120, the storage device1130, and/or the interface 1140.

The storage device 1130 may store data and/or commands and the like. Theinterface 1140 may perform the function of transferring data to thecommunication network or receiving data from the communication network.The interface 1140 may be in wired or wireless form. For example, theinterface 1140 may include an antenna or wired/wireless transceiver andthe like.

Although it is not illustrated, the image system may further include ahigh-speed DRAM and/or SRAM, as an operation memory for improving theoperation of the camera module 1110 and the processor 1120.

The image system may be applied to a personal digital assistant (PDA), aportable computer, a web tablet, a wireless phone, a mobile phone, orall electronic products that can transmit and/or receive informationunder the wireless circumstance.

In some embodiments, the image system may be applied to the mobilephones. For example, in the image system 3000 of FIG. 16, the processor1120 and the interface 1140 may be disposed on the board 1100 andprovided in the form of a single board. The camera module 1110 maycommunicate with the processor 1120 via the interface 1140. Thesubstrate 1100 may be, for example, a substrate of a mobile phone. Theprocessor 1120 may be, for example, an application processor AP built ina mobile phone.

While the present inventive concept has been particularly illustratedand described with reference to exemplary embodiments thereof, it willbe understood by those of ordinary skill in the art that various changesin form and detail may be made therein without departing from the spiritand scope of the present inventive concept as defined by the followingclaims. The exemplary embodiments should be considered in a descriptivesense only and not for purposes of limitation.

What is claimed is:
 1. An image processing device comprising: a cameramodule comprising a first camera module comprising a first sensorconfigured to sense light from an object and generate a first imagesignal, and a first storage device storing first calibration data forthe first camera module, a second camera module comprising a secondsensor configured to sense light from the object and generate a secondimage signal, and a second storage device storing second calibrationdata for the second camera module, and a third camera module comprisinga third sensor configured to sense light from the object and generate athird image signal, and a third storage device storing third calibrationdata for the third camera module; and an application processor separatedfrom the first, second and third camera modules, wherein the applicationprocessor includes a first camera serial interface configured to receivethe first image signal from the first camera module, a second cameraserial interface configured to receive the second image signal from thesecond camera module, wherein the first and second image signals arecolor image signals, a third camera serial interface configured toreceive the third image signal from the third camera module, and whereinthe application processor is configured to select one of the secondimage signal and the third image signal as a selected image signal basedon a selection signal, and generate an output image based on the firstimage signal and the selected image signal, wherein the selection signalincludes a zoom selection signal which is changed in response to a scaleratio.
 2. The image processing device of claim 1, wherein theapplication processor generates the output image using the firstcalibration data in the first storage device, and one of the secondcalibration data in the second storage device and the third calibrationdata in the third storage device.
 3. The image processing device ofclaim 2, wherein the first camera module has a first lens having a firstangle, the second camera module has a second lens having a second angle,and the third camera module has a third lens having a third angledifferent from the second angle.
 4. The image processing device of claim3, wherein the third angle is wider than the second angle.
 5. The imageprocessing device of claim 4, wherein the first lens is disposed betweenthe second lens and the third lens.
 6. The image processing device ofclaim 1, wherein the application processor further includes a firstsignal processor connected to the first camera serial interface andconfigured to perform first image signal processing on the first imagesignal to generate first processed image data; a second signal processorconnected to the second camera serial interface and configured toperform second image signal processing on the second image signal togenerate second processed image data; and a third signal processorconnected to the third camera serial interface and configured to performthird image signal processing on the third image signal to generatethird processed image data.
 7. The image processing device of claim 6,wherein the application processor further includes a selector configuredto receive the first processed image data and one of the secondprocessed image data and the third processed image data, and to outputone of the first processed image data, the second processed image data,and the third processed image data responsive to the zoom selectionsignal.
 8. The image processing device of claim 1, wherein theapplication processor further includes a depth information generationunit configured to receive the first image signal and one of the secondimage signal and the third image signal, and to output depthinformation, a first output image generation unit configured to receivethe first image signal and the depth information and to generate a firstoutput image.
 9. The image processing device of claim 8, wherein theapplication processor further includes a second output image generationunit configured to receive the first image signal and one of the secondimage signal and the third image signal, and, to receive disparityinformation from the depth information generation unit and, to generatea second output image.
 10. An image processing device comprising: afirst camera module comprising a first sensor configured to generate afirst image signal, a first lens and a first storage device storingfirst calibration data for the first camera module; a second cameramodule comprising a second sensor configured to generate a second imagesignal, a second lens and a second storage device storing secondcalibration data for the second camera module; a third camera modulecomprising a third sensor configured to generate a third image signal, athird lens and a third storage device storing third calibration data forthe third camera module, wherein the first lens is disposed between thesecond lens and the third lens; and an application processor separatedfrom the first, second and third camera modules and configured togenerate an image output of an object based on the first image signaland one of the second image signal and the third image signal, whereinthe application processor includes a first camera serial interfaceconfigured to receive the first image signal from the first cameramodule, a second camera serial interface configured to receive thesecond image signal from the second camera module, a third camera serialinterface configured to receive the third image signal from the thirdcamera module, wherein the application processor includes a first imagegenerator configured to output a first image output signal by processingthe first image signal using the first calibration data in the firststorage device, and to output a second image output signal by processingone of the second image signal and the third image signal using one ofthe second calibration data in the second storage device and the thirdcalibration data in the third storage device, and wherein the firstimage generator includes an adjusting unit configured to receive thefirst image output signal and the second image output signal and toperform a rectification for correcting a distortion of the first andsecond image output signals.
 11. The image processing device of claim10, wherein the application processor is configured to generate theimage output based on depth information of the object.
 12. The imageprocessing device of claim 11, wherein the application processor isconfigured to provide an out-focusing effect to the image output basedon the depth information of the object.
 13. The image processing deviceof claim 10, wherein the application processor is configured to generatethe image output responsive to a zoom selection signal, the first imagesignal, and the one of the second image signal and the third imagesignal.
 14. The image processing device of claim 10, wherein the firstcamera module is disposed between the second camera module and the thirdcamera module.
 15. The image processing device of claim 14, wherein athird angle of the third lens is different from a second angle of thesecond lens.
 16. The image processing device of claim 15, wherein thethird angle is wider than the second angle.
 17. The image processingdevice of claim 16, wherein a first angle of the first lens is differentfrom the second angle.